Optimal Provision of Regulation Service Reserves Under Dynamic Energy Service Preferences

We propose and solve a stochastic dynamic programming (DP) formulation of the optimal provision of regulation service reserves (RSR) by controlling dynamic demand preferences in smart buildings. A major contribution over past dynamic pricing work is that we pioneer the relaxation of static, uniformly distributed utility of demand. In this paper we model explicitly the dynamics of energy user preferences leading to a non-uniform and time varying probability distribution of demand utility. More explicitly, we model active and idle duty cycle appliances in a smart building as a closed queuing system with price-controlled arrival rates into the active appliance queue. Focusing on cooling appliances, we model the utility associated with the transition from idle to active as a non-uniform time varying function. We (i) derive an analytic characterization of the optimal policy and the differential cost function, and (ii) prove monotonicity and convexity properties. This enables us to propose and implement a smart assisted value iteration algorithm as well as an approximate DP solution based on the aforementioned functional approximations and properties. Numerical results demonstrate the validity of the solution techniques and the computational advantages of the proposed near optimal approximate-DP solutions for realistic, large-state-space problems.